Image correction device and image correction method

ABSTRACT

An image correction apparatus includes: an image input part to which an image including plural character element rows are input; a row detection part for detecting a predetermined character element row from the plural character element rows; a correction amount calculating part for performing calculation of a position correction amount in a column direction with respect to each pixel column on the predetermined character element row; and a position correction part for correcting a position of each pixel column of the image so as to move it in the column direction based on the position correction amount calculated with respect to each pixel column in a predetermined direction.

TECHNICAL FIELD

The present invention relates to an image correction apparatus and animage correction method for correcting a slant or meandering of acharacter row and the like produced in an image obtained by shooting anoriginal such as a document by a hand scanner and the like.

BACKGROUND ART

Conventionally, various technologies for performing characterrecognition by shooting an original such as a document by a scanner andthe like and performing OCR (Optical Character Recognition) using theshot image have been proposed.

Specifically, in an apparatus adopting a system in which an user moves arelatively compact scanner such as a hand scanner over the original toshoot an image, depending on the handling of the user, it is difficultto scan the scanning direction at the time of shooting in a constantdirection relative to the alignment direction of characters etc. of theoriginal. On this account, sometimes a slant or meandering occurs in theresultant shot image compared to the original. As a result, when thedegree of the slant or meandering is large, there is a problem that thecharacters can not be cut out correctly and the character recognitionrate is reduced.

As a method for correcting such slant or meandering of an image, forexample, a method for generating a projection profile by making acharacter image into image data consisting of an aggregate of pixelsarranged in a two-dimensional manner, binarizing brightness values withrespect to each pixel to form two dimensional binarized image data,setting a large number of scan lines mutually parallel to each pixel toperform scanning, accumulating data representing the character image ofthe binarized image data with respect to each scan line, and obtaining adistribution in a direction perpendicular to the scan lines with respectto the accumulated value, and for obtaining an amount of rotationcorrection based on the dispersion value of the projection profile hasbeen proposed (for example, see Publication of Japanese Patent No.3108979).

However, in the image correction method as described above, since shiftis performed in units of character elements, the meandering with respectto each character can be corrected but the distortion of the characterelement itself can not be corrected, and thus, there is a problem thatsuitable character recognition can not be performed in the subsequentprocessing such as OCR.

DISCLOSURE OF THE INVENTION

The invention is objected to provide an image correction method capableof correcting a slant or meandering of a character element row as wellas correcting the distortion of the character element itself in light ofthe problems.

An image correction apparatus of the invention is characterized byincluding: an image input part to which an image including pluralcharacter element rows are input; a row detection part for detecting apredetermined character element row from the plural character elementrows; a correction amount calculating part for performing calculation ofa position correction amount in a column direction with respect to eachpixel column on the predetermined character element row; and a positioncorrection part for correcting a position of each pixel column of theimage so as to move it in the column direction based on the positioncorrection amount calculated with respect to each pixel column in apredetermined direction.

According to such constitution, correction is performed on all of thepixel columns constituting the image to move them in the columndirection, and thereby, not only the meandering and slant of characterelement rows can be corrected, but also the distortion of each characterelement can be corrected.

Further, the row detection part may have a histogram generating part forgenerating an integrated histogram along a row direction of the image,and detect the longest character element row based on the integratedhistogram.

According to such constitution, by the simple processing of calculatingthe integrated histogram of the image, the predetermined characterelement row the position correction amount of which should be calculatedcan be selected while suppressing the load on the computation part.

Furthermore, the row detection part may have a pixel position extractingpart for extracting a pixel position where a value of the integratedhistogram generated in the histogram generating part becomes themaximum, and detect the longest character element row based on the pixelposition.

According to such constitution, the predetermined character element rowcan be detected by detecting character element row including the pixelposition where the value of the integrated histogram becomes themaximum.

Moreover, the row detection part may have a range identifying part foridentifying a pixel position range where the value of the integratedhistogram falls within a predetermined range as the longest characterelement row from the pixel position extracted in the pixel positionextracting part.

According to such constitution, the predetermined character element rowcan be identified simply and clearly by determining the predeterminedrange in advance.

Further, the correction amount calculating part may have an end positiondetection part for detecting an end position in the column directionwith respect to each character element of the predetermined characterelement row, and calculate the position correction amount based on adisplacement amount of the end position with respect to each characterelement row.

According to such constitution, since the processing for detecting theend position in the column direction with respect to each characterelement of the predetermined character element row is performed, theload on the computing part can be reduced compared to the case wherecomputation processing is performed with respect to all of the characterelement rows in the image.

Furthermore, the correction amount calculating part may calculate thedisplacement amount based on an envelope curve connecting the endpositions detected by the end position detection part with respect toeach character element.

According to such constitution, the displacement amount can becalculated by the simple processing of calculating the envelope curve onthe predetermined character element row with respect to each characterelement.

Further, the image correction apparatus may include: an image input partto which an image including plural character element rows are input; ahistogram generating part for generating an integrated histogram along arow direction of the image; a pixel position extracting part forextracting a pixel position where a value of the integrated histogramgenerated in the histogram generating part becomes the maximum; a rangeidentifying part for identifying a pixel position range where the valueof the integrated histogram falls within a predetermined range as arange of the longest character element row from the pixel positionextracted in the pixel position extracting part; an end positiondetection part for detecting an end position in the column direction inthe image with respect to each character element of the longestcharacter element row; a position correction amount calculating part forcalculating a displacement amount of the end positions with respect toeach character element row based on an envelope curve connecting the endpositions detected by the end position detection part with respect toeach character element row; and a position correction part forcorrecting the image with respect to each pixel column so as to move itin the column direction based on the position correction amount.

According to such constitution, not only the meandering and slant ofcharacter element rows can be corrected, but also the distortion of eachcharacter element can be corrected.

Next, the image correction apparatus of the invention is characterizedby including: an image input part to which a first image includingplural character element rows are input; an expanded row generating partfor generating a second image including plural expanded rows byexpanding the first image in a row direction; a starting positiondetection part for detecting a starting position of the expanded row inthe column direction with respect to each pixel column of the secondimage; a correction amount calculating part for calculating a positioncorrection amount in a column direction with respect to each pixelcolumn of the second image; and a position correction part forcorrecting a position of each pixel column of the first image so as tomove it in the column direction based on the position correction amount.

According to such constitution, since the starting position of theexpanded row in the column direction is detected to determine the rangeof the pixel positions constituting the character element row, there isa lower possibility that the character element rows superpose and thecharacter element rows can be separated with higher accuracy compared tothe case where the range of existence of the entire character isdetected. Therefore, even when the character element rows are shotslanted to some extent, the slant as well as the meandering can becorrected simultaneously.

Further, the second image may be a binarized image having brightnessvalues expressed by a value of 0 or 1.

According to such constitution, used amount of memory can be reduced,processing can be performed rapidly, and the load on the computing partcan be reduced, and thereby, mounting on the portable informationequipment and the like can be made easier.

Furthermore, the starting position detection part may perform detectionof the starting position of the expanded row in the column direction by,while moving a pixel of interest in the column direction, detecting abrightness value of the pixel of interest with respect to each pixelcolumn, and, when equal to or more than a predetermined number of pixelshaving brightness values of 0 continue, setting a position where thepixel having the brightness value of 0 is detected for the first time asthe starting position.

According to such constitution, since the possibility that the noiseinformation due to dirt or the like is regarded as a character elementcan be made lower, more suitable image correction can be performed.

Moreover, the starting position detection part may perform detection ofthe starting position of the expanded row in the column direction withrespect to each of plural expanded rows, and the correction amountcalculating part may calculate the position correction amount based onan average value of a starting position distribution of each of theplural expanded rows in the column direction.

According to such constitution, the influence by the character such as“j” or “p” protruding more downward than other characters is reduced,and thereby, more suitable image correction can be performed.

Next, the image correction apparatus of the invention is characterizedby including: an image input part to which a binarized first imageincluding plural character element rows are input; an expanded rowgenerating part for generating a second image including plural expandedrows by expanding the first image in a row direction; a startingposition detection part for, while moving a pixel of interest in acolumn direction, detecting a brightness value of the pixel of interestwith respect to each pixel column of the second image, and, when equalto or more than a predetermined number of pixels having brightnessvalues of 0 continue, detecting a position where the pixel having thebrightness value of 0 is detected for the first time as a startingposition of the expanded row; a correction amount calculating part forcalculating a position correction amount with respect to each pixelcolumn of the second image based on an average value of a startingposition distribution of the plural expanded rows in the columndirection; and a position correction part for correcting the first imagewith respect to each pixel column so as to move it in the columndirection based on the position correction amount.

According to such constitution, since the starting position of theexpanded row is detected to determine the range of the lower endposition of the character element row, there is a lower possibility thatthe character element rows superpose and the character element rows canbe separated with higher accuracy compared to the case where the rangeof existence of the entire character is detected. Therefore, even whenthe character element rows are shot slanted to some extent, the slant aswell as the meandering can be corrected simultaneously.

Next, information equipment and a cellular phone device including theimage correction apparatus of the invention may be provided.

According to such constitution, since characters and the like, theslant, meandering, and distortion of character elements of which areimage corrected for easy character recognition etc. can be input,especially in the information equipment and cellular phone deviceequipped with the character recognition such as an OCR function, theaccuracy of character reading can be made higher.

Next, the image correction method of the invention is characterized byincluding: a first step for detecting a predetermined character elementrow from an image including plural character element rows; a second stepfor calculating a position correction amount with respect to each pixelcolumn of the predetermined character element row; and a third step forcorrecting the image with respect to each pixel column so as to move itin the column direction based on the position correction amount.

According to such constitution, correction is performed on all of thepixel columns constituting the image to move them in the columndirection, and thereby, not only the meandering and slant of characterelement rows can be corrected, but also the distortion of each characterelement can be corrected.

Further, the image correction method of the invention may include: afirst step for generating a second image including plural expanded rowsby expanding a first image including plural character element rows in arow direction; a second step for detecting a starting position of theexpanded row in a column direction with respect to each pixel column ofthe second image; and a third step for correcting a position of thefirst image so as to allow starting positions of the expanded row in thecolumn direction to align with each other based on information of thestarting positions of the expanded row in the column direction.

According to such constitution, since the starting position of theexpanded row in the column direction is detected to determine the rangeof the lower end positions constituting the character element row, thereis a lower possibility that the character element rows superpose and thecharacter element rows can be separated with higher accuracy compared tothe case where the range of existence of the entire character isdetected. Therefore, even when the character element rows are shotslanted to some extent, the slant as well as the meandering can becorrected simultaneously.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart showing processing steps of an image correctionmethod in the first embodiment of the invention.

FIG. 2 is a block diagram showing an example of the constitution of animage correction apparatus in the first embodiment of the invention.

FIG. 3A shows an example of an original image in the first embodiment ofthe invention.

FIG. 3B shows an example of a slant corrected image in the firstembodiment of the invention.

FIG. 4 shows an example of a horizontal integrated histogram in thefirst embodiment of the invention.

FIG. 5 shows an example of a detection result of a black pixel lower endposition in the first embodiment of the invention.

FIG. 6 shows an example of a detection result of a vertical positiondisplacement amount in the first embodiment of the invention.

FIG. 7 shows a meandering corrected image in the first embodiment of theinvention.

FIG. 8 is a flowchart showing the processing procedure of an imagecorrection method in the second embodiment of the invention.

FIG. 9 is a block diagram showing an example of the constitution of animage correction apparatus in the second embodiment of the invention.

FIG. 10 shows an example of an original image in the second embodimentof the invention.

FIG. 11 shows an example of a binarized image in the second embodimentof the invention.

FIG. 12 is a flowchart showing the steps of the horizontal expansionprocessing in the second embodiment of the invention.

FIG. 13 is a schematic diagram for explanation of the contents of thehorizontal expansion processing in the second embodiment of theinvention.

FIG. 14 shows an example of a horizontally expanded image in the secondembodiment of the invention.

FIG. 15 is a flowchart showing the steps of starting position detectionprocessing of the expanded character row in the second embodiment of theinvention.

FIG. 16 shows a histogram of the starting position of the expandedcharacter row in the second embodiment of the invention.

FIG. 17 shows a distribution of the starting position of the expandedcharacter row in the second embodiment of the invention.

FIG. 18 shows a relative displacement amount distribution of thestarting point of the expanded character row in the second embodiment ofthe invention.

FIG. 19 shows an example of a corrected image in the second embodimentof the invention.

FIG. 20A is a front view of a cellular phone device equipped with theimage correction apparatus in the third embodiment of the invention.

FIG. 20B is a perspective view of a main part showing an end part wherethe image correction apparatus is mounted in the third embodiment of theinvention.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the invention will be described in detailusing the drawings.

(First Embodiment)

First, as the first embodiment of the invention, an image correctionapparatus and an image correction method of the invention will bedescribed.

FIG. 1 is a flowchart showing processing steps of an image correctionmethod in the first embodiment of the invention. Further, FIG. 2 showsan example of the constitution of image correction apparatus 40 forexecuting the image correction method in the first embodiment of theinvention.

As shown in FIG. 2, image correction apparatus 40 in the firstembodiment of the invention includes image input part 1 for reading anoriginal such as a character and graphic and inputting an image thereof,CPU 2 connected to image input part 1 for performing various kinds ofprocessing as described below based on the image input to image inputpart 1, storage means 3 such as a frame memory for storing the imageinput to image input part 1, and display part 5 for displaying variouskinds of computed result information or necessary information to anuser.

As image input part 1, a device selected from devices such as an opticaldevice used for the publicly known hand scanner can be used.

Storage means 3 is connected to CPU 2 and, as a storage medium thereof,the publicly known flash memory and the like can be used.

Display part 5 can arbitrarily be selected from the publicly knowndisplay devices, for example, an LCD (Liquid Crystal Display), an EL(Electro-Luminescent), a CRT (Cathode Ray Tube), etc.

Next, the processing steps of the image correction method in the firstembodiment of the invention will be described according to FIG. 1.

First, an image shot in image input part 1 (hereinafter, referred to as“original image”) is loaded (developed) in storage means 3 via the CPU 2(S1).

An example of original image 10 is shown in FIG. 3A. Original image 10shown in FIG. 3A is an image obtained by combining partial images shotby image input part 1 and the direction of its character row is slantedby being greatly affected by the angular difference between the scanningdirection when the user manually scans and the row direction of thecharacters. Further, since the trail as image input part 1 is moved overthe original is meandered relative to the character row direction,original image 10 shown in FIG. 3A is meandered. Note that, in theembodiment, original image 10 is an aggregate of pixels arranged in atwo-dimensional manner and a monochrome image of pixels each having amultilevel (256 levels of gray) brightness value.

Further, in the embodiment, as image shooting means in image input part1, a CCD of 256×16 pixels is used. Furthermore, as storage means 3 forimage development, a frame memory of the horizontal direction relativeto the sheet surface of FIG. 3A (lateral)×the vertical directionrelative to the sheet surface (longitudinal)=1000×400 pixels is used.

Then, CPU 2 executes processing of correcting the slant of the entireimage relative to original image 10 stored in storage means 3 (S2). Theinvention is not for restricting anything with respect to the processingof correcting the slant, but publicly known methods can be usedtherefor. For example, in JP-A-1-156887, a method for rotating originalimage 10 by rotating original image 10 to plural angles, calculating ahistogram along the row direction, and determining an angle at which thewidth of the histogram becomes the minimum as an angle to which originalimage 10 should be rotated is disclosed. Such method can be used, or anyother publicly known method may be used.

In FIG. 3B, an example of slant corrected image 11 on which such slantcorrection processing has been performed is shown. Here, the example inwhich slant corrected image 11 is constituted by black pixels having adensity value of “1” and white pixels having a density value of “0”,i.e., so-called binarized image is shown. Slant corrected image 11 hasfour character rows (A to D in FIG. 3). Note that, in the specification,the character row refers to the connection of character elements alongthe direction in which characters are written (for example, the X axisdirection in FIG. 3B).

By the comparison between slant corrected image 11 and original image10, it is seen that the slant of the entire image has been corrected,but the above described meandering remains. As below, a method forcorrecting meandering of an image in the embodiment will be described.

Turning to FIG. 1 again, CPU 2 performs calculation of an integratedhistogram on slant corrected image 11 stored in storage means 3 bycounting the number of black pixels (adding density values) alonghorizontal direction with respect to each horizontal line with respectto each vertical pixel position in FIG. 3B (S3).

Note that, in the specification, the horizontal pixel alignment in theimage data constituting original image 10 is referred to as “horizontalline”, and the vertical pixel alignment is referred to as “verticalline”.

FIG. 4 shows a result obtained by calculating a horizontal integratedhistogram with respect to slant corrected image 11 shown in FIG. 3B. InFIG. 4, the lateral axis indicates the position of a pixel along thevertical direction of the image (indicates an example in which the upperleft corner relative to the sheet surface in FIG. 3B is set as anoriginal point O), and along the longitudinal axis, the numbers of blackpixels are plotted.

As seen from FIG. 4, in the embodiment, it is known that the calculatedblack pixel histogram is divided into four crest parts (A to D). Thesecrest parts correspond to the above described four character rows (A toD) in slant corrected image 11 shown in FIG. 3B.

Subsequently, CPU 2 calculates the vertical pixel position where thenumber of black pixels becomes the maximum from the horizontalintegrated histogram that has been calculated in the above describedstep S3, and regards the crest part including the maximum value as thelongest character row (S5). In the embodiment, since crest part Bincludes the vertical pixel position where the number of black pixelsbecomes the maximum, character row B is regarded as the longestcharacter row.

Further, CPU 2 determines the vertical width of the correspondingcharacter row with respect to crest part B, which has been regarded asthe longest (S5). Specifically, a vertical pixel position range wherethe number of black pixels becomes a predetermined ratio, R % relativeto the maximum value (the range shown by W in FIG. 4) is determined aslongest character row range W. R is set to a value that enablesseparation from adjacent character rows. Practically, separation ofcharacter rows can be ensured by setting R to about 30%.

By the processing steps heretofore, longest character row range W alongthe vertical direction for determining an amount to be meanderingcorrected can be determined.

Next, in the vertical pixel position range determined as longestcharacter row range W, CPU 2 scans slant corrected image 11 shown inFIG. 3B from the original point along the X axis direction sequentiallywith respect to each one vertical line from the lower side relative tothe sheet surface in FIG. 3B, and determines the position where a blackpixel is detected for the first time as a black pixel lower endposition. The processing is performed with respect to all of thevertical lines (S6).

FIG. 5 shows a result obtained by detecting the black pixel lower endposition with respect to all of the vertical lines of slant correctedimage 11 shown in FIG. 3B. For simplicity of description, in FIG. 5, theslant corrected image 11 is displayed in a superimposing manner.

As seen from FIG. 5, it is known that the black pixel lower end positionat the lowest end is detected with respect to each character elementthat constitutes the character row regarding the character row B(strictly, longest character row range W).

Turning to FIG. 1, then, CPU 2 calculates an envelope curve connectingthe lowest points of the black pixel lower end positions relative to thesheet surface from the calculation result shown in FIG. 5 and determinesit as a vertical displacement amount (S7). Specifically, assuming thatthe area sandwiched between the vertical lines with the black pixellower end positions of “0” as one character element, the lowest blackpixel lower end position is detected with respect to each areaconstituting each character element, the adjacent black pixel lower endpositions with respect to each character are connected with straightlines, and the vertical position displacement amount to be verticallycorrected is calculated with respect to each vertical line. An exampleof the detection result of the vertical position displacement amount isshown in FIG. 6.

Then, CPU 2 calculates the amount to be vertically displaced withrespect to each vertical line based on the vertical positiondisplacement amount shown in FIG. 6, vertically displaces with respectto each vertical line stored in storage means 3, and ends the processing(S8). By the way, the system in which display 5 displays the meanderingcorrected image to the user after this step may be adopted.

Thus, according to the image correction method or the image correctionapparatus of the invention, since the correction is performed bydetecting the black pixel lower end position with respect to each arearegarded as one character, forming an envelope curve that connects theadjacent black pixel lower end positions with respect to characterelement, and performing correction by displacing all of the verticallines based on the vertical position displacement amount, the distortionof character element itself can be improved.

By the way, in the above description, the example in which therespective processing steps from step S2 to step S8 are realized withsoftware is shown. However, the invention is not limited to that, and atleast one step from step S1 to step S8 may be realized with hardwarehaving a function of each step.

In FIG. 7, meandering corrected image 13 processed according to theimage correction apparatus and the image correction method described inthe embodiment is shown. As seen from FIG. 7, as compared to slantcorrected image 11 shown in FIG. 3B, the degree of meandering of thecharacter rows is drastically reduced. As described in the embodiment,it can be confirmed that the meandering of the entire image can becorrected by determining the amount to be corrected based on the longestcharacter row and performing correction with respect to vertical linesthat constitute the entire image.

As described above, according to the image correction apparatus or theimage correction method of the embodiment, since the correction isperformed with respect to the entire image by detecting the longestcharacter row by horizontal histogram calculation and detecting thedisplacement amount to be corrected with respect to the character row,the processing can be performed more rapidly compared to the case wherethe displacement correction is performed with respect to all of thecharacter lines that constitute the image.

Next, a specific example in which the recognition accuracy of theoriginal is improved by the image correction apparatus or the imagecorrection method of the embodiment.

The calculation of the correct reading ratio is performed by performingOCR processing in CPU 2 based on the image stored in storage means 3 andcalculating the rate of the resulting correctly recognized characters.As a sample, recognition is performed using 20 business cards at random.The number of characters shot and subjected to OCR is 390 characters oftelephone numbers and 1026 characters of mail addresses and URLs.

First, in the case of the telephone numbers in the business cards,relative to the correct reading rate when no correction is performed,about 20% of the correct reading rate can be improved by the imagecorrection apparatus or the image correction method of the embodiment.

Further, with respect to the mail addresses and URLs in the businesscards, relative to the correct reading rate when no correction isperformed, about 25% of the correct reading rate can be also improved bythe image correction apparatus or the image correction method of theembodiment, and higher correct reading rate can be obtained.

Furthermore, with respect to the entire of the telephone numbers, mailaddresses and URLs in the business cards, relative to the correctreading rate when no correction is performed, about 23% of the correctreading rate can be also improved when the image correction of theinvention is performed. It is conceivable that this is because themeandering of the character rows can be corrected as well as thedistortion of the image of the character itself can be correctedaccording to the image correction apparatus or the image correctionmethod of the invention.

(Second Embodiment)

Next, as the second embodiment of the invention, another example of theimage correction apparatus or the image correction method of theinvention will be described.

FIG. 8 is a flowchart showing the processing procedure of an imagecorrection method in the second embodiment of the invention. Further,FIG. 9 is a block diagram showing an example of the constitution ofimage correction apparatus 130 for realizing the image correction methodin the second embodiment of the invention.

As shown in FIG. 9, image correction apparatus 130 in the secondembodiment of the invention includes image input part 101 for reading anoriginal such as a character and graphic and inputting an image thereof,CPU 102 connected to image input part 101 for performing various kindsof processing as described below based on the image input to image inputpart 101, first storage means 103 for storing the image input to imageinput part 101 as image information of pixels arranged in atwo-dimensional manner, further, second storage means 104 for storingthe obtained image as a result of various kinds of computationprocessing by CPU 102, and display part 105 connected to CPU 102 fordisplaying various kinds of computed result information or necessaryinformation to the user.

Note that, the image information of pixels in the invention refers tovarious kinds of information such as brightness information, colorinformation, and density information with respect to pixels thatconstitute the image, and, in the embodiment, the brightness informationof pixels is used.

As image input part 101, a device arbitrarily selected from devices suchas an optical device used for the publicly known hand scanner can beused.

First storage means 103 and second storage means 104 are frame memory,respectively, and, as a storage medium thereof, the publicly knownmedium such as a flash memory can be used.

Display part 105 can arbitrarily be selected from the publicly knowndisplay devices, for example, an LCD (Liquid Crystal Display), an EL(Electro-Luminescent), a CRT (Cathode Ray Tube), etc.

Next, the processing steps when image correction apparatus 130 in theembodiment of the invention performs image correction will be describedaccording to FIG. 8.

First, image shot in image input part 101 (hereinafter, referred to as“original image”) 110 is developed as brightness information as imageinformation of pixels arranged in a two-dimensional manner in firststorage means 103 via CPU 102 (S10).

An example of original image 110 is shown in FIG. 10. Original image 110shown in FIG. 10 is an image obtained by combining partial images shotby image input part 101 and, since the trail when image input part 101is moved over the original is meandered relative to the character row,original image 110 shown in FIG. 10 is meandered.

Note that, in the embodiment, as described above, original image 110 isan aggregate of pixels arranged in a two-dimensional manner and amonochrome image of pixels each having a multilevel (256 levels of gray)brightness value.

Further, in the embodiment, as shooting means of image input part 101, aCCD of 256×16 pixels is used, and, as first storage means 103 for imagedevelopment, a frame memory of the horizontal direction relative to thesheet surface (lateral)×the vertical direction relative to the sheetsurface (longitudinal)=1000×400 pixels in FIG. 10 is used.

Then, CPU 102 executes binarization processing of storing either valueof 0 (black) or 1 (white) with respect to each pixel as brightnessinformation using the publicly known method on original image 110 storedin first storage means 103 (S11). By the binarization processing,reduction in used amount of memory and speeding up of the processing aremade possible, and the load on CPU 102 can be suppressed.

In FIG. 11, an example of binarized image 111 subjected to suchbinarization processing. The pixels that appear to be black are pixelsin which the brightness value “0” is stored as brightness information,and the pixels that appear to be white are pixels in which thebrightness value “1” is stored as brightness information. By the way, inthe invention, any publicly known method can be used as the binarizationprocessing method of images, there is no limitation.

Turning to FIG. 8, then, CPU 102 performs horizontal expansionprocessing for horizontally expanding the black pixel part of the imagewith respect to binarized image 111 stored in first storage means 103(S12).

The horizontal expansion processing will be further described. FIG. 12is a flowchart showing the steps of the horizontal expansion processing(expanded row generating means).

In FIG. 12, first, CPU 102 sets an arbitrary pixel in first storagemeans 103 as a pixel of interest. Practically, a pixel located at themost end of binarized image 111, i.e., a pixel located on the originalpoint O at the upper left of the sheet surface of FIG. 11 is set as thepixel of interest. Then, the brightness value of the pixel of interestis detected (S21).

Subsequently, CPU 12 judges whether the brightness value of the pixel ofinterest is 0 (black) or not (S22), if the brightness value of the pixelof interest is 0 (black), the brightness value of the pixels in apredetermined range forward and rearward of the corresponding pixel ofinterest along the processing direction in second storage means 104 isset to 0 (black) (S23). On the other hand, if the brightness value ofthe pixel of interest is not 0 (is 1), this processing is not performed.

Step S23 will be described using FIG. 13. In FIG. 13, if the brightnessvalue of a pixel of interest A in the first storage means 103 is 0(black), with respect to the pixels in a predetermined range forward andrearward along the processing direction (in the embodiment, total 41pixels including 20 pixels each for forward and rearward and the pixelof interest), the brightness values of the corresponding pixels insecond storage means 104 are uniformly set to brightness value=0 (black)regardless of their original brightness values. Note that, it ispractically desirable that the predetermined range includes about 20pixels forward and rearward.

Turning to FIG. 12, subsequently, CPU 102 judges whether the processingwith respect to all of the pixels that constitute first storage means103 is completed or not (S24), and, if the processing with respect toall of the pixels is completed, the processing is ended. On the otherhand, if the processing with respect to all of the pixels is notcompleted, the pixel of interest is shifted along the processingdirection in FIG. 11 (S25), and the processing returns to the step fordetecting the brightness value of the pixel of interest (S21).

By such processing, horizontally expanded image 13 as shown in FIG. 14can be obtained on the second storage means 104. FIG. 14 shows anexample of horizontally expanded image 113 in the embodiment.

As shown in FIG. 14, horizontally expanded image 113 has plural expandedcharacter rows. Note that, here, the expanded character row (expandedrow) refers to a block of continuous pixels having brightness values of0, i.e., black pixels. In the embodiment, four expanded character rowsLA to LD of horizontally expanded image 113 correspond to four characterrows A to D in binarized image 111, respectively.

Here, turning to FIG. 8, after the horizontal expansion processing(S12), CPU 102 detects the starting position of the black pixels alongthe vertical direction with respect to each expanded character row(S13). That is, CPU 102 scans from the leftmost vertical pixel column ofhorizontally expanded image 113 relative to the sheet surface in FIG. 14(referring to the pixel alignment along the vertical direction (rowdirection) in the image) with respect to each vertical pixel column fromlower side of FIG. 14 toward the Y axis direction and detects the pixelposition where the black pixel is detected at the first time(hereinafter, referred to as “starting position”) in the case where thecontinuation of the black pixels (brightness value=0) continues over thepredetermined number of pixels with respect to each expanded characterrow. That is, in horizontally expanded image 113 of the embodiment, thestarting positions of four expanded character rows LA to LD are detectedwith respect to each vertical pixel column.

The detection method of the expanded character row will be describedusing FIG. 15. FIG. 15 is a flowchart showing the steps of startingposition detection processing (starting position detection means) of theexpanded character row in the embodiment of the invention.

In FIG. 15, first, CPU 102 sets a pixel of interest with respect to anarbitrary vertical pixel column of horizontally expanded image 113stored in second storage means 104, and detects the brightness valuewith respect to each pixel of interest while shifting the pixel ofinterest (S31). Note that, practically, the setting of the pixel ofinterest is performed from the lowermost pixel in FIG. 14.

Then, CPU 102 judges whether equal to or more than the predeterminednumber of the continuous black pixels are detected or not (S32), and, ifequal to or more than the predetermined number of the continuous blackpixels are detected, the position of the pixel where the black pixelstars for the first time is stored as a starting position of theexpanded character row (S34). On the other hand, if less than thepredetermined number of the continuous black pixels are detected, thecontinuation is regarded not as an expanded character row but noiseinformation, and the pixel of interest is shifted (S36) and theprocessing proceeds to the starting position detection processing of thenext expanded character row. Note that, it is practically desirable thatthe predetermined number includes approximately 20 pixels.

Then, whether the pixel of interest reaches the upper end of thevertical pixel column or not is judged (S35), if it reaches, theprocessing is ended.

The processing as described above is performed with respect to all ofthe vertical pixel columns (entire screen) that constitute horizontallyexpanded image 113. By such processing, a short continuation of blackpixels is regarded as noise information, and thereby, only theinformation of the expanded character rows constituted by character rowscan be drawn and processed and the constitution hardly affected by thenoise information can be realized.

As described above, a result (histogram) obtained by plotting thevertical positions and the integrated values of the accumulated numberof starting positions in the respective vertical positions from theobtained starting points of all of the expanded character rows as aresult of performing starting position detection processing of theexpanded character rows on all of the vertical pixel columns is shown inFIG. 16. FIG. 16 shows a distribution of the expanded character rowstarting points in the embodiment of the invention, and shows the rangeof the lower end positions of character rows.

Turning to FIG. 8, CPU 102 determines the range of the lower endposition of each character row (as an example, the range W in FIG. 16)using the relationship shown in FIG. 16. The processing (row separatingmeans) is obtained by detecting widths of crests having areas equal toor more than a certain value from the histogram shown in FIG. 16(hereinafter, the processing is referred to as “grouping”) (S14). In theembodiment, four crests exist as shown in FIG. 16, and these correspondto the ranges of existence of the starting positions of expandedcharacter rows LA to LD, respectively.

Note that, in the step S14, in the case where the area of the crest ofthe histogram is less than the predetermined value, considered as noiseinformation, the information of the integrated value is neglected. Bysuch system, suitable starting position detection of the expandedcharacter row with less influence of noise information can be performed.

FIG. 17 shows a distribution of the starting position with respect toeach expanded character row when the grouping of the starting positionsof the expanded character rows in the embodiment of the invention isperformed. The lateral axis indicates the horizontal position of theexpanded image in FIG. 14, and the longitudinal axis indicates thevertical position. In this manner, the lines show the starting points offour expanded character rows LD, LC, LB, and LA sequentially from thetop, i.e., the lower end positions.

Thus, according to the image correction method and the image correctionapparatus of the invention, since the ranges of the lower positions ofthe characters are not superposed and easily separated by the groupingof the starting positions of the expanded character rows, the separationof the character rows can be performed with high accuracy.

Turning to FIG. 8, then, CPU 102 calculates displacement amount to bevertically corrected on four expanded character rows LA to LD in FIG. 14with respect to all of the vertical pixel columns (S15), and then,calculates the average value of the displacement amounts of fourexpanded character rows LA to LD (S16).

FIG. 18 shows a relative displacement amount distribution of therespective vertical pixel columns in the embodiment of the invention.The lateral axis indicates the horizontal position and the longitudinalaxis indicates the relative displacement amount obtained by calculatingthe average value with respect to all of the character rows. By therelationship shown in FIG. 18, the displacement amount to be verticallycorrected is determined with respect to each vertical pixel column.

In FIG. 8, CPU 102 (movement computing part) moves the respectivevertical pixel columns of binarized image 111 stored in first storagemeans 103 along the vertical direction by the displacement amount basedon the displacement amount average value, and ends the processing (S17).

In FIG. 19, an example of corrected image 115 as a result of performingimage correction on original image 110 by image correction apparatus 130in the embodiment of the invention is shown.

As seen from FIG. 19, compared to original image 110, it is known thatboth of the degrees of the slant and the meandering of the characterrows are corrected. Thus, the slant and meandering are corrected byusing the image correction apparatus of the invention, and thereby,reduction in recognition rate in the subsequent processing such as OCRcan be suppressed.

By the way, in the above description, the example in which therespective processing steps from step S11 to step S17 are realized withsoftware is shown. However, the invention is not limited to that, and atleast one step from step S10 to step S17 may be realized with hardwarehaving a function of each step.

According to the image correction apparatus of the invention, since theaverage value of the displacement amounts calculated with respect toeach character row is used as the displacement amount to be corrected,even when there is a character element protruding downward, for example,such as “j” or “p” in a certain character row, the system in which therow is hardly and adversely affected by the character element can berealized.

Note that the image correction apparatus or the image correction methodof the invention is not limited to the constitution described in theembodiment. For example, a system for performing correction on all ofthe vertical pixel columns constituting the image by referring to onlythe displacement amount to be corrected, which is calculated withrespect to the character row including the largest integrated valuebased on the histogram of the starting points of the expanded characterrows shown in FIG. 16 may be adopted. This is because the character rowincluding the largest integrated value normally means the longestcharacter row, and the displacement amount to be corrected is detectedwith respect to the character row and correction is performed, andthereby, a major part of the entire image can be corrected. Practically,by such constitution, the slant and meandering of the image can also bepreferably corrected. Further, the processing can be performed morerapidly by calculating the starting positions with respect to all of thecharacter rows constituting the image compared to the case where thedisplacement correction is performed based on their average value.

Image correction is performed by using image correction apparatus 130 orthe image correction method in the embodiment of the invention usingrandom 20 business cards as samples. The number of characters subjectedto OCR is 390 characters of telephone numbers and 1026 characters ofmail addresses and URLs. After the image correction, OCR processing isperformed by CPU 102 based on the image stored in first storage means103, and, as a result of calculating the rate of the correctlyrecognized characters, in the entire of the telephone numbers, mailaddresses and URLs in the business cards, relative to the correctreading rate when no correction is performed, about 25% of the correctreading rate can be improved.

By the way, in the embodiment of the invention, the function of displaypart 105 is not described specifically, however, by arranging thedisplay part so as to display the acquired original image, the binarizedimage, etc., or messages such as an error message or an messagerepresenting necessary input contents to the user, a user-friendlyapparatus constitution can be realized.

Note that, in the first embodiment or the second embodiment of theinvention, the example in which the slant and meandering of therespective characters, numerals, etc. on the image read by the imagecorrection apparatus are corrected is described, however, the imagecorrectable by the image correction apparatus of the invention is notlimited to that. Needless to add, for example, the image correctionapparatus or the image correction method of the invention can correctthe slant, meandering, distortion, or the like of the read image withrespect to an original in which information such as barcodes andgraphics in place of or in addition to characters, numerals and the like(such information is generically named as “character element” in thespecification) aligned in a direction on the image.

(Third Embodiment)

Next, as the third embodiment of the invention, an information equipmentincluding the image correction apparatus or the image correction methodof the invention will be described.

Since the used amount of memory can be reduced, processing can beperformed rapidly, and the load on the computing part (CPU) can bereduced by using the image correction apparatus or the image correctionmethod of the invention, mounting on portable information equipment andthe like becomes easier. The example of mounting image correctionapparatus 40, 130 on the information equipment such as a cellular phonedevice is shown in FIG. 20.

FIG. 20 is an outside drawing of cellular phone device 150 equipped withimage correction apparatus 130, and FIG. 20A is a front view thereof andFIG. 20B is a perspective view of a main part showing an end part whereimage correction apparatus 130 is mounted.

Cellular phone device 150 has constitution in which image correctionapparatus 130 is built-in in the publicly known cellular phone deviceincluding antenna part 151, speaker part 152, display part 105 such asan LCD, key part 154, and microphone part 155.

By mounting the image correction apparatus of the invention, a surfacefor reading information represented by the density of the characters,graphics, etc. of image input part 101 of image correction apparatus 130is provided on the lower surface of cellular phone device 150, andthereby, very user-friendly image correction apparatus 130 built-incellular phone device 150 can be provided.

Note that, in the embodiment, an example in which image correctionapparatus 130 described in the second embodiment is mounted on cellularphone device 150 has been described, however, as a matter of course,mounting image correction apparatus 40 described in the first embodimentcan exert the same effect.

As described above, since document information such as URLs andtwo-dimensional barcodes can be read and subjected to processing such asOCR by the cellular phone device equipped with the image correctionapparatus of the invention, multifunctional information equipment suchas a cellular phone device that has never been existed can be provided.

Needless to add, the information equipment here is not limited to theabove described cellular phone device, but includes publicly knownvarious kinds of information equipment such as a digital camera, compactpersonal computer, and PDA (personal digital assistant).

Industrial Applicability

As described above, an image correction apparatus and an imagecorrection method according to the invention has an advantage that theslant or meandering of character rows can be corrected and thedistortion of the character element itself can also be corrected, andare useful as an image correction apparatus and an image correctionmethod and the like for correcting the slant or meandering of characterrows and the like produced in an image obtained by shooting an originalsuch as a document by a hand scanner and the like.

1. An image correction apparatus comprising: an image input part towhich an image including plural character element rows are input; a rowdetection part for detecting a predetermined character element row fromthe plural character element rows; a correction amount calculating partfor performing calculation of a position correction amount in a columndirection with respect to each pixel column on the predeterminedcharacter element row; and a position correction part for correcting aposition of each pixel column of the image so as to move it in thecolumn direction based on the position correction amount calculated withrespect to each pixel column in a predetermined direction.
 2. The imagecorrection apparatus according to claim 1, wherein the row detectionpart has a histogram generating part for generating an integratedhistogram along a row direction of the image, and detects the longestcharacter element row based on the integrated histogram.
 3. The imagecorrection apparatus according to claim 2, wherein the row detectionpart has a pixel position extracting part for extracting a pixelposition where a value of the integrated histogram generated in thehistogram generating part becomes the maximum, and detects the longestcharacter element row based on the pixel position.
 4. The imagecorrection apparatus according to claim 3, wherein the row detectionpart has a range identifying part for identifying a pixel position rangewhere the value of the integrated histogram falls within a predeterminedrange as a range of the longest character element row from the pixelposition extracted in the pixel position extracting part.
 5. The imagecorrection apparatus according to claim 1, wherein the correction amountcalculating part has an end position detection part for detecting an endposition in the column direction with respect to each character elementof the predetermined character element row, and calculates the positioncorrection amount based on a displacement amount of the end positionwith respect to each of the character element rows.
 6. The imagecorrection apparatus according to claim 5, wherein the correction amountcalculating part calculates the displacement amount based on an envelopecurve connecting the end positions detected by the end positiondetection part with respect to each of the character elements.
 7. Animage correction apparatus comprising: an image input part to which animage including plural character element rows are input; a histogramgenerating part for generating an integrated histogram along a rowdirection of the image; a pixel position extracting part for extractinga pixel position where a value of the integrated histogram generated inthe histogram generating part becomes the maximum; a range identifyingpart for identifying a pixel position range where the value of theintegrated histogram falls within a predetermined range as a range ofthe longest character element row from the pixel position extracted inthe pixel position extracting part; an end position detection part fordetecting an end position in the column direction in the image withrespect to each character element of the longest character element row;a position correction amount calculating part for calculating adisplacement amount of the end positions with respect to each of thecharacter elements based on an envelope curve connecting the endpositions detected by the end position detection part with respect toeach of the character elements; and a position correction part forcorrecting the image with respect to each pixel column so as to move itin the column direction based on the position correction amount.
 8. Animage correction apparatus comprising: an image input part to which afirst image including plural character element rows are input; anexpanded row generating part for generating a second image includingplural expanded rows by expanding the first image in the row direction;a starting position detection part for detecting a starting position ofthe expanded row in the column direction with respect to each pixelcolumn of the second image; a correction amount calculating part forcalculating a position correction amount in the column direction withrespect to each pixel column of the second image; and a positioncorrection part for correcting a position of each pixel column of thefirst image so as to move it in the column direction based on theposition correction amount.
 9. The image correction apparatus accordingto claim 8, wherein the second image is a binarized image havingbrightness values expressed by a value of 0 or
 1. 10. The imagecorrection apparatus according to claim 9, wherein the starting positiondetection part performs detection of the starting position of theexpanded row in the column direction by, while moving a pixel ofinterest in the column direction, detecting a brightness value of thepixel of interest with respect to each of the pixel columns, and, whenequal to or more than a predetermined number of pixels having brightnessvalues of 0 continue, setting a position where the pixel having thebrightness value of 0 is detected for the first time as the startingposition.
 11. The image correction apparatus according to claim 10,wherein the starting position detection part performs detection of thestarting position of the expanded row in the column direction withrespect to each of plural expanded rows, and the correction amountcalculating part calculates the position correction amount based on anaverage value of a starting position distribution of each of the pluralexpanded rows in the column direction.
 12. An image correction apparatuscomprising: an image input part to which a binarized first imageincluding plural character element rows are input; an expanded rowgenerating part for generating a second image including plural expandedrows by expanding the first image in a row direction; a startingposition detection part for, while moving a pixel of interest in acolumn direction, detecting a brightness value of the pixel of interestwith respect to each pixel column of the second image, and, when equalto or more than a predetermined number of pixels having brightnessvalues of 0 continue, detecting a position where the pixel having thebrightness value of 0 is detected for the first time as a startingposition of the expanded row; a correction amount calculating part forcalculating a position correction amount with respect to each of thepixel columns of the second image based on an average value of astarting position distribution of the plural expanded rows in the columndirection; and a position correction part for correcting the first imagewith respect to each of the pixel columns so as to move it in the columndirection based on the position correction amount.
 13. Informationequipment comprising: an image input part to which an image includingplural character element rows are input; a row detection part fordetecting a predetermined character element row from the pluralcharacter element rows; a correction amount calculating part forperforming calculation of a position correction amount in a columndirection with respect to each pixel column on the predeterminedcharacter element row; and a position correction part for correcting aposition of each pixel column of the image so as to move it in thecolumn direction based on the position correction amount calculated withrespect to each pixel column in a predetermined direction.
 14. Acellular phone device comprising: an image input part to which an imageincluding plural character element rows are input; a row detection partfor detecting a predetermined character element row from the pluralcharacter element rows; a correction amount calculating part forperforming calculation of a position correction amount in a columndirection with respect to each pixel column on the predeterminedcharacter element row; and a position correction part for correcting aposition of each pixel column of the image so as to move it in thecolumn direction based on the position correction amount calculated withrespect to each pixel column in a predetermined direction.
 15. An imagecorrection method comprising: a first step for detecting a predeterminedcharacter element row from an image including plural character elementrows; a second step for calculating a position correction amount withrespect to each pixel column of the predetermined character element row;and a third step for correcting the image with respect to each pixelcolumn so as to move it in the column direction based on the positioncorrection amount.
 16. An image correction method comprising: a firststep for generating a second image including plural expanded rows byexpanding a first image including plural character element rows in a rowdirection; a second step for detecting a starting position of theexpanded row in a column direction with respect to each pixel column ofthe second image; and a third step for correcting a position of thefirst image so as to allow starting positions of the expanded row in thecolumn direction to align with each other based on information of thestarting positions of the expanded row in the column direction.